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In the printed circuit board (PCB) industry, surface-mount technology is heavily utilized for the mounting of 
electronic components. Currently, most PCB inspection methods consider solder joint defects only, without checking the 
position error for components. A practical inspection method that also considers the existing PCB components themselves 
would be highly beneficial. In this work, an algorithm is developed to check the existence of surface-mount devices by the 
binocular disparity method. Binocular disparity is generally used to extract depth information from the two-dimensional 
retinal images in stereopsis. However, this 3D stereo disparity technique could be used to quickly detect the components on 
a PCB -extra components as well as missing components. This work includes an application demonstration of the algorithm 
for PCB defect inspection. 
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INTRODUCTION 

In the printed circuit board (PCB) industry, surface-mount technology is heavily utilized for the mounting of 
electronic components recently. As a result, the electronic components rely on the solder joint to provide the electrical 
connection information to the PCBs; therefore, the quality of the solder joint can be critical to the quality of the electronic 
components, especially. The size of electronic components become smaller for every period, therefore, automatic Optical 
Inspection of solder joints has been a critical issue for quality control in PCB assembly as AOI has the enormous potential 
of completely automating human visual inspection procedures. 

A survey by Moganti shown the algorithms and techniques for the automated inspection of printed circuit boards 
are examined. A classification tree for these algorithms is presented and the algorithms are grouped according to this 
classification. This survey concentrates mainly on image analysis and fault detection strategies; these also include 
state-of-the-art techniques. Wang introduces the development of an automated visual inspection system for printed circuit 
boards (PCBs). It utilizes an elimination- subtraction method which directly subtracts the template image from the 
inspected image, and then conducts an elimination procedure to locate defects in the PCB. Guerra and Villalobos 
developed the techniques using laser triangulation method to develop a three-dimensional automated inspection system for 
SMDs. Oyeleye and Lehtihet developed a classification algorithm and optimal feature selection methodology for 
implementation on an automated solder defect inspection system. Lin and Chau presented a two-steps inspection method 
for image inspection and classification. Zen and Ma developed the methodologies for locating and identifying components 
on a printed circuit board used for surface mount device inspection using layered illumination. 
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HARDWARESETUP 

The inspection method is different from the most tradition methods, using 2D images and low-angle ring direct 
lighting illumination to get the reflective light from solder joint surface. For this work, the illumination system is simpler. 
The only need for the lighting is to having the diffuse lighting. The diffusing light provides more significant 3D matching 
features for the correlation between two images. 

Figure 1 illustrated the top and side view cameras, the inspection target on the different inspection layers. 
The inspection target may not layon the layer located on the interaction center of the top and side cameras, where zis equal 
to 0. Due to the printed circuit board's warpage, the inspection target may be on the higher layer(z=h, h>0) or lower layer 
(z=-h)instead of the default inspection layer (z=0), where the five cameras (one top and four side cameras) have the same 
co-center position of the five camera field of views(FOVs). The inspection target may appear on the different layer because 
of the printed circuit board's warpage. The warpage is caused by temperature or some internal or external forces during the 
manufacturing processes. While Z is not equal to 0, the images need to be relocated to Z=0 by rectifying the side images. 
The warpage is detected by corresponding features histogram. 




Figure 1: Hardware Setup for Top View and Side View of Cameras 

The five cameras are setup shown as Figure 2. There are four side cameras to provide the different direction's 
binocular disparity images. In this research, a special gauge calibration tool is developed to check the precision of the 
inspection system. 



Figure 2: Top Camera and Four Side Cameras 

METHODOLOGY AND ALGORITHM 

In this research, the binocular disparity technique is developed to detect the heights of electronic components, the 
missing components, misalignment, and wrong orientation of IC chips or wrong parts. Binocular disparity technique is 
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usually used to process the 3D object reconstruction and the space position detection. In this research, it's a new 
application used to detect the PCB's warpage and the heights of electronic components. 

Binocular Disparity 

Binocular Disparity method refers to the difference locations of the object seen by human's left and right eyes, 
where human could detect the 3D location of the object in space by the difference scenes between eyes. Figure 3 illustrated 
the object sareshow on the camera 1 and camera 2. The functions of the cameras are very similar to human eyes. 
The objects P and Q are represented on the different locations of the cameras, where P is seen in pi on the left camera and 
seen in pr on the right camera, while Q is seen in pi on the left camera and seen in qr on the right camera. The distance Z 
could be calculated by the Equation 1 and 2. 




m T ► camera 

Figure 3: Binocular Disparity 



In this work, the heights of the electronics components could be detected by one pair cameras. However, some 
components maybe hidden by other components on the printed circular board. Therefore, for the reliability of the 
inspection, in the research there are four side cameras, front, rear, left and right cameras. Each pair could provide the height 
information for the double check the inspection results. 

Corresponding Matching 

Corresponding matching method is used in this work to find the warpage. The features points of the images are 
listed to match to that of another image, for example, the result shown on Figure 4. The corresponding matching method 
help find the offset distance between the pair images. In addition, in this work, the pair images were shot from different 
angles' cameras, the corresponding matching method also reveals the height information of components. 
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Figure 4: Corresponding Images 

Inspection Flow and Algorithm 

The inspection process is illustrated in Figure 5. The inspection system grabs a set of five images from top and 
four side cameras. The images need to be rectified by rectified matrix to correct the distortions. After the projective images 
are corrected, it uses stereo disparity technology and image corresponding method to obtain the board's warpage. 
The detected warpage could help the system to make better rectified image, and finally the images could be performed 
background subtraction to find the missing component, wrong direction, and extra components detection. The components' 
heights could be detected by stereo disparity method. 
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Figure 5: Inspection Process 



ON-LINE INSPECTION 



For the traditional 2D inspection methods, the template images need to be trained before the on-line inspection 
process. However, in currently industry, mass production use a lot of substance components to support the assembly line. 
Use the prepared trained template causes some troubles while there are couple different alternative supplies. The same 
functional components may have different appearances. In this research, the template images are provided by top camera, 
and another disparity images are provided by angle view cameras simultaneously. Therefore, the template is trained on- 
line, and it's more practical for real- world's mass productions. 
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Figure 6: Binocular Disparity d 

In Figure 6, the Point O is the image center of top camera as well as that of side camera. While the target 
component with height, h, is inspected, the top camera keeps having the same sensor position for Point H and O. However, 
for side camera, point h is shown on the different sensor position from the center of image sensor. The difference between 
point H and O in image sensor is d. Here, heights of components could be calculated by 
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d*r 



r = 



sin (0) sin (0) 

Real world (mm) 
sensor pixel 

Where r is the real world scale (mm) per sensor pixel 

In this work, the figure 7 shows the inspection result for electronic components (Chip components). 
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Figure 7: The Inspection Results. The PCB's Warpage is -180 um (-9 Pixels, 20um per Pixel), and the Height of 

Component is 659 um (33.0 Pixel) 

Currently, most PCB inspection methods only use the light tracking to check the slop of the solder joint to 
measure the condition of solder joint. However, the work will not work if the surface-mount device components are not on 
the board or there are any extra components. In this work, an algorithm using binocular disparity method is developed to 
check if there any extra components or lacks on the board. 
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Figure 8: (a) Image from Top Camera (b). Image from Front Camera 

Figure 8 (a) and (b) illustrated the same component captured by top and front camera, separately. After the two 
images are rectified to be stereo images pair, the subtraction of the grey levels of the two images not only reveal the height 
information of the component, but also could be used to detect the component existence or missing, too. The subtraction of 
the grey level is illustrated as in Figure 9. Where, the blue and red color showed the difference of the intensity between 
Figure8 (a) and (b). The size of blue or red color areas represents the height of the component and show if there are the 
components existed. Where, the color pair of blue and red colors represented simultaneously. The color pair could increase 
the inspection reliability. 




Figure 9: Agray Level Subtraction 

SUMMARY 

In the research, the on-line mass production method is provided to do the inspection of SMD components for 
SMT mass production. This work is good at detecting the missing components, wrong directions, and extra components. 
In addition, the components' heights could be obtained by stereo disparity method without extra construct light or special 
pattern projected to inspection target, which will reduce a lot of cost for inspection system. 
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